The invention relates to video enhancement technology. Video enhancement is a process that improves the image values of an input digital video by reducing noise such as camera noise, or distortions such as compression artifacts or by increasing the image sharpness.
Conventional video enhancement systems take advantage of the space and time redundancy between pixel values to suppress noise and distortions by means of averaging filters. To take into account movements in videos, motion-compensated filters average pixel values along estimated motion trajectories. To reduce the memory requirements in a video system, recursive filters are often used. However, when the estimated motion vectors are not accurate, time recursive filters produce strong visual artifacts in the processed video.
An efficient adaptive spatial image filtering is implemented by thresholding wavelet coefficients determined from the image. This is equivalent to adaptively averaging image values over neighborhoods of varying sizes, which are adapted to the local image content. In regular regions, the image values are averaged over large domains whereas they are averaged over smaller domains near edges or irregular textures.
To take advantage of the adaptive averaging capabilities of the wavelet transform together with the time redundancy of a video, motion-compensated time recursive filtering has been proposed to reduce the noise in wavelet coefficients. The time averaging is performed over a scale that depends upon the recursive filter parameters. The efficiency of these noise reduction algorithms depends on the ability to automatically and appropriately adjust the recursive filter parameters. Ad-hoc procedures have been developed in “Wavelet video denoising with regularized multiresolution motion estimation”, F. Jin, et al., EURASIP Journal on Applied Signal Processing, Volume 2006, pp. 1-11, January 2007, and in “Wavelet-domain Video Denoising Based on Reliability Measures”, V. Zlokolica, et al., IEEE Transactions on Circuits and Systems for Video Technology, Vol. 16, No. 8, pp. 993-1007, August 2006. These procedures adjust the recursive parameters based on some measurement of the motion reliability and an estimation of the noise level. The adjustment of the recursive parameters may not provide an automatic multiscale transform in time that relies on a stable signal representation, and thus does not guarantee a robust noise reduction algorithm.
WO 2007/059795 discloses implementing space-time noise reduction algorithms with bandlet transforms constructed with multiscale linear combinations of wavelet coefficients along spatial or temporal geometric flows. Thresholding bandlet coefficients provides a robust procedure to adaptively perform a space-time signal averaging that adjusts the scale in time and in space to the local variations of the signal content. However, for videos, state of the art multiscale bandlet averaging requires storing, reading and writing at least one frame per scale, which is expensive in terms of memory storage and bandwidth.
Some video enhancement systems can sharpen the image by amplifying high frequencies. Non-linear sharpness enhancement methods have been applied to wavelet coefficients by amplifying these coefficients depending on the scale. A wavelet enhancement system can integrate noise reduction and enhancement by setting to zero smaller coefficients and amplifying larger ones. However such enhancement systems do not take advantage of the time regularity of videos. A video bandlet sharpness enhancement procedure can be implemented with the same type of enhancement operators as wavelet enhancement, but applied to bandlet coefficients, thus taking advantage of the time redundancy of videos. Like for noise reduction, state of the art bandlet systems require writing many intermediate frames.
There is a need for a video enhancement system capable of obtaining reliable results for various kinds of video signals, using a fairly stable basis or frame for decomposing the video images. It is also desirable to find an appropriate procedure to adapt multiscale recursive filtering parameters in order to compute multiscale bandlet coefficients that are thresholded or amplified to perform an adaptive enhancement.